Papers with PEFT strategy
Preserving Pre-trained Representation Space: On Effectiveness of Prefix-tuning for Large Multi-modal Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large multi-modal models (LMMs) are revolutionizing the way machines interact with the world, unlocking new possibilities across multi-dimensional applications. |
| Approach: | They propose a parameter-efficient fine-tuning strategy that combines both . they find that parameter tuning methods distort the feature representation space . |
| Outcome: | The proposed strategy preserves representation space while limiting performance on downstream tasks. |
Debiasing the Fine-Grained Classification Task in LLMs with Bias-Aware PEFT (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to mitigate label biases such as retraining, post-hoc adjustment, and parameter-efficient fine-tuning fail to address prediction propensity and discriminative ability biase. |
| Approach: | They propose a bias-aware optimization framework that incorporates two distinct label balance constraints with a PEFT strategy targeting an intermediate layer to mitigate this issue. |
| Outcome: | The proposed approach outperforms or matches the performance of full-parameter fine-tuning and LoRA, achieving superior results with lower perplexity. |